Balancing a Pipeline

Reduction of a matrix to triangular form plays a crucial role in the solution of linear equations. In this chapter, I analyze a pipeline algorithm for Householder reduction (Brinch Hansen 1990). The pipeline is folded several times across an array of processors to achieve approximate load balancing. The pipeline inputs, transforms, and outputs a matrix, column by column. During the computation, the columns are distributed evenly among the processors. The computing time per column decreases rapidly from the first to the last column. So, the performance of the algorithm is limited mainly by the order in which the columns are distributed among the processors. The simplest idea is to store a block of columns with consecutive indices in each processor (Ortega 1988). Block storage performs poorly because it assigns the most time-consuming columns to a single processor and leaves much less work for other processors. It is much better to distribute the columns cyclically among the processors, so that each processor holds a similar mixture of columns. This storage pattern is called wrapped mapping or scattered decomposition (Ortega 1988, Fox 1988). A third method is reflection storage where the columns are distributed one at a time by going back and forth across the processors several times (Ortega 1988). The folded pipeline combines block and reflection storage. On a Computing Surface with 25 transputers, the Householder pipeline achieves an efficiency of 81% for a 1250×1250 real matrix.